Image processing device and image processing method

ABSTRACT

There is provided an image processing device including a depth acquisition unit configured to acquire depth information of an object included in a captured image, an image combining unit configured to combine the object with an image object having depth information according to each piece of the depth information, and an effect processing unit configured to execute effect processing according to the depth information of the object.

BACKGROUND

The present disclosure relates to an image processing device and animage processing method.

Recently, image processing devices that process captured images havebeen widely spread. For example, an image processing device thatextracts an object from a captured image and combines the object withanother image object is disclosed in Japanese Patent ApplicationPublication No. 2006-128754.

SUMMARY

When the object is combined with another image object as describedabove, it is desirable that the object be in an appropriate range forthe other image object. For example, when the image object is an imageobject of a celebrity, it is desirable that a depth of the object beclose to a depth of the image object of the celebrity so that the objectis viewed along with the celebrity.

However, it may be difficult to recognize whether or not the object isin the appropriate depth range for the image object just at a glance ofa composite image of the object and the image object. For example, whenthere is a small object at a shallow depth and when there is a largeobject at a deep depth, it is difficult to determine whether or not theobject is in the appropriate depth range according to a size of theobject in the composite image because there is no large difference inthe size of the object in the composite image.

It is desirable to provide a novel and improved image processing deviceand image processing method that can provide notification so that it canbe easily determined whether or not an object is in an appropriaterange.

In accordance with an embodiment of the present disclosure, there isprovided an image processing device including a depth acquisition unitconfigured to acquire depth information of an object included in acaptured image, an image combining unit configured to combine the objectwith an image object having depth information according to each piece ofthe depth information, and an effect processing unit configured toexecute effect processing according to the depth information of theobject.

In accordance with another embodiment of the present disclosure, thereis provided an image processing method including acquiring depthinformation of an object included in a captured image, combining theobject with an image object having depth information according to eachpiece of the depth information, and executing effect processingaccording to the depth information of the object.

In accordance with the embodiments of the present disclosure describedabove, notification can be provided so that it can be easily determinedwhether or not an object is in an appropriate range.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram illustrating a configuration of animage processing system in accordance with an embodiment of the presentdisclosure;

FIG. 2 is an explanatory diagram illustrating the extraction of a humanimage from a captured image;

FIG. 3 is an explanatory diagram illustrating a combining process forthe human image;

FIG. 4 is a functional block diagram illustrating a configuration of animage processing device 20 in accordance with an embodiment of thepresent disclosure;

FIG. 5 is an explanatory diagram illustrating an appropriate depth rangeof an object;

FIG. 6 is an explanatory diagram illustrating a composite image when adepth of the human image is too deep;

FIG. 7 is an explanatory diagram illustrating a composite image when thedepth of the human image is too shallow;

FIG. 8 is an explanatory diagram illustrating a composite image when thehuman image is separated from a foreground object;

FIG. 9 is a flowchart illustrating an operation of an image processingsystem in accordance with an embodiment of the present disclosure; and

FIG. 10 is an explanatory diagram illustrating an example of a hardwareconfiguration of the image processing device 20.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

In addition, in this specification and the drawings, a plurality ofstructural elements having substantially the same function and structuremay be distinguished by adding different letters after the samereference numerals. However, when it is not necessary to particularlydistinguish the plurality of structural elements having substantiallythe same function and structure, the plurality of structural elementsare denoted by the same reference numeral only.

In addition, description of the present disclosure will be given in theorder of following items.

1. Basic Configuration of Image Processing System

2. Configuration of Image Processing Device

3. Operation of Image Processing System

4. Hardware Configuration

5. Conclusion

<1. Basic Configuration of Image Processing System>

Technology by the present disclosure can be practiced in various formsas will be described hereinafter as one example. In addition, an imageprocessing device in accordance with the embodiment of the presentdisclosure includes:

A. a depth acquisition unit (a communication unit 220) configured toacquire depth information of an object included in a captured image;

B. an image combining unit (an image combining unit 250) configured tocombine the object with an image object having depth informationaccording to each piece of the depth information; and

C. an effect processing unit 260 configured to execute effect processingaccording to the depth information of the object.

Hereinafter, first, the basic configuration of the image processingsystem including the above-described image processing device will bedescribed with reference to FIGS. 1 to 3.

FIG. 1 is an explanatory diagram illustrating the configuration of theimage processing system in accordance with an embodiment of the presentdisclosure. As illustrated in FIG. 1, the image processing system inaccordance with the embodiment of the present disclosure includes animaging device 10 and an image processing device 20.

(Imaging Device)

The imaging device 10 images an object to acquire a captured image.Specifically, the imaging device 10 includes an imaging optical systemsuch as an imaging lens or a zoom lens and an imaging element such as acharge coupled device (CCD) or a complementary metal oxide semiconductor(CMOS).

The imaging optical system forms an object image on an imaging plane ofthe imaging element by condensing light emitted from the object. Theimaging element converts the object image formed by the imaging opticalsystem into an electrical image signal. Here, the imaging elementincludes a red (R) component light-receiving element, a green (G)component light-receiving element, and a blue (B) componentlight-receiving element. The captured image including RGB information isobtained according to a function of each light-receiving element.

In addition, the imaging device 10 in accordance with the embodiment ofthe present disclosure acquires depth information indicating a distancebetween the imaging device 10 and the object. This depth information canbe acquired in units of pixels in the captured image.

Further, the imaging device 10 extracts an image part of the object byrecognizing the object from the captured image based on theabove-described depth information. For example, the imaging device 10extracts a human image from the captured image based on the depthinformation, human skeleton information, and the like. The imagingdevice 10 transmits the extracted human image to the image processingdevice 20 along with the depth information. Hereinafter, this point willbe specifically described with reference to FIG. 2.

FIG. 2 is an explanatory diagram illustrating the extraction of a humanimage from a captured image. The captured image 70 illustrated in FIG. 2includes the human image 82 and a background image. Here, because thereis a depth difference between a region of the human image 82 and aregion of the background image, the imaging device 10 may extractcandidates for the human image based on a depth distribution of eachpixel of the captured image 70, and further determine a candidateconsistent with a human skeleton pattern as the human image. Thereby, asin a processing image 72 illustrated in FIG. 2, the human image 82 canbe extracted from the captured image 70.

Although an example in which the object is a human in this specificationwill be described, the object is not limited to the human. For example,the object may be another object like a flower or an animal. Inaddition, although an example in which the human image is extracted fromthe captured image based on the depth information has been describedabove, the human image can be extracted according to image recognitionof a pattern matching a human.

(Image Processing Device)

The image processing device 20 receives a human image and depthinformation of the human image from the imaging device 10, and combinesthe human image with another image object using the depth information.For example, the image processing device 20 combines a background imageand a foreground image with the human image. Hereinafter, this pointwill be specifically described with reference to FIG. 3.

FIG. 3 is an explanatory diagram illustrating a combining process for ahuman image. As illustrated in FIG. 3, the image processing device 20generates a composite image 90 by combining a background image 74, aforeground image 76 including a celebrity as a foreground object 84, anda processing image 72 including a human image 82. Here, because theforeground object 84 has depth information, the image processing device20 can express an anteroposterior relationship between the foregroundobject 84 and the human image 82 based on depth information of theforeground object 84 and the human image 82.

As described above, it is possible to obtain a composite image as if ahuman is in another space by combining the human imaged by the imagingdevice 10 with another image object.

Although a personal computer (PC) is illustrated as an example of theimage processing device 20 in FIG. 1, the image processing device 20 isnot limited to the PC. For example, the image processing device 20 maybe an information processing device such as a home video processingdevice (a digital versatile disc (DVD) recorder, a video cassetterecorder, or the like), a personal digital assistant (PDA), a home gamemachine, a home appliance, or the like. In addition, the imageprocessing device 20 may be an information processing device such as aportable telephone, a personal handy-phone system (PHS), a portablemusic player device, a portable video processing device, a portable gamemachine, or the like.

In addition, although an example in which the imaging device 10 extractsthe human image 82 has been described above, the image processing device20 can extract the human image 82. Further, a function of theabove-described image processing device 20 can be implemented in theimaging device 10, and the imaging device 10 can function as the imageprocessing device.

(Point of View of Present Disclosure)

Incidentally, it is desirable that a depth of the human image 82 beclose to a depth of the foreground object 84 of the celebrity so thatthe human is viewed along with the celebrity when the human image 82 iscombined with the foreground object 84 of the celebrity as describedabove.

However, it may be difficult to recognize whether or not the object isin an appropriate depth range for the image object just at a glance of acomposite image of the object and the image object. For example, becausethere is no difference in a size of the object in the composite imagewhen there is a small object at a shallow depth and when there is alarge object at a deep depth, it is difficult to determine whether ornot the object is in the appropriate depth range by the size of theobject in the composite image.

An embodiment of the present disclosure has been made in view of theabove-described circumstances. In accordance with the embodiment of thepresent disclosure, notification can be provided so that it can beeasily determined whether or not the object is in an appropriate range.Hereinafter, the above-described image processing device 20 inaccordance with the embodiment of the present disclosure will bedescribed in detail.

<2. Configuration of Image Processing Device>

FIG. 4 is a functional block diagram illustrating a configuration of theimage processing device 20 in accordance with an embodiment of thepresent disclosure. As illustrated in FIG. 4, the image processingdevice 20 in accordance with the embodiment of the present disclosureincludes the communication unit 220, a storage unit 230, a display unit240, the image combining unit 250, and the effect processing unit 260.

The communication unit 220 is an interface (I/F) with the imaging device10, and receives a human image or depth information of the human imagefrom the imaging device 10. The communication unit 220 may establish awireless or wired connection to the imaging device 10.

The storage unit 230 is a storage medium that stores a background imageand a foreground image to be combined with the human image. In furtherdetail, depth information is set for a foreground object included in theforeground image. Although depth information may be set for every pixelconstituting the foreground object, it is mainly assumed that one pieceof representative depth information is set for the foreground object inthis specification. The storage unit 230 may store a composite imagegenerated by the image combining unit 250.

The above-described storage section 230 may include a storage mediumsuch as a nonvolatile memory, a magnetic disk, an optical disc, or amagneto-optical (MO) disc. An example of the nonvolatile memory may be aflash memory, a secure digital (SD) card, a micro SD card, a universalserial bus (USB) memory, an electrically erasable programmable ROM(EEPROM), or an erasable programmable ROM (EPROM). In addition, themagnetic disk may be a hard disk, a disc-shaped magnetic body disk, orthe like. In addition, the optical disc may be a compact disc (CD), adigital versatile disc (DVD), a Blu-ray disc (BD) (registeredtrademark), or the like.

The display unit 240 displays the composite image generated by the imagecombining unit 250. Because this composite image is a composite image ofthe human image obtained by imaging and a virtual background image andforeground image, a user can have visual contact with the compositeimage displayed on the display unit 240, thereby experiencing a sensethat the user is in a space different from an actual space.

The image combining unit 250 combines the human image received by thecommunication unit 220 with the background image and the foregroundimage stored in the storage unit 230. The background image and theforeground image to be combined may be selected by the user's operation.In addition, the background image and the foreground image may besupplied from an external device such as a server on a network insteadof the storage unit 230.

Here, for every pixel constituting the human image, the image combiningunit 250 expresses an anteroposterior relationship between the humanimage and the foreground object based on the depth information of thehuman image and the depth information of the foreground object includedin the foreground image. Thereby, as illustrated in FIG. 3, it ispossible to generate a realistic composite image. The foreground objectmay be a still image or a moving image. When the foreground object isthe moving image, the depth information of the foreground object isvaried with movement of the foreground object.

When the image combining unit 250 generates the composite image, theeffect processing unit 260 executes effect processing if a depth of thehuman image is not in an appropriate range. Thereby, notification can beprovided so that it can be easily determined and intuitively understoodwhether or not the human is in the appropriate depth range. Hereinafter,the appropriate depth range and the effect processing will be morespecifically described.

(Appropriate Depth Range)

As illustrated in FIG. 3, it is desirable that a depth of a human imagebe close to a depth of a foreground object of a celebrity so that thehuman is viewed along with the celebrity. That is, it is appropriatethat the depth of the human image is in a predetermined range based onthe depth of the foreground object. Thus, the effect processing unit 260may treat a range of less than a threshold value Dth from the foregroundobject as an appropriate depth range as illustrated in FIG. 5.

Although the same threshold value Dth is illustrated as front and rearthreshold values of the foreground object in FIG. 5, the front and rearthreshold values may be different. In addition, the threshold value Dthmay be a different value according to the foreground image. For example,when the depth of the foreground object is deep, an influence by thesame depth difference is small as compared to when the depth of theforeground object is shallow. Thus, the effect processing unit 260 mayset the threshold value Dth to a larger value as the depth of theforeground object is deeper.

(Effect Processing)

The effect processing unit 260 does not perform the effect processingwhen the depth of the human image is in the above-described appropriatedepth range, that is, when a difference between the depth of the humanimage and the depth of the foreground object is less than the thresholdvalue Dth. On the other hand, the effect processing unit 260 executesthe effect processing for providing notification that the human is notin the appropriate depth range when the depth of the human image is outof the above-described appropriate depth range, that is, when thedifference between the depth of the human image and the depth of theforeground object is greater than or equal to the threshold value Dth.Hereinafter, a specific example of the effect processing will bedescribed.

FIG. 6 is an explanatory diagram illustrating a composite image 92 whena depth of a human image is too deep. When a depth of a human image 82is deeper than a depth of a foreground object 84 by the threshold valueDth or more, the effect processing unit 260 makes the human image 82semi-transparent by applying alpha processing to the human image 82 asin the composite image 92 illustrated in FIG. 6. In addition, the effectprocessing unit 260 adds an alarm mark 88 for attention to the compositeimage 92 as illustrated in FIG. 6.

The effect processing unit 260 may perform uniform effect processing onthe human image by treating a mean value of a depth of each pixel of thehuman image or a depth of a body part as a representative depth andcomparing the representative depth of the human image to the depth ofthe foreground object.

Alternatively, the effect processing unit 260 may compare the depth ofthe human image to the depth of the foreground object in units of pixelsof the human image. In this case, the alpha processing may be applied toonly part of the human.

FIG. 7 is an explanatory diagram illustrating a composite image 94 whena depth of a human image is too shallow. When a depth of a human image82 is shallower than a depth of a foreground object 84 by the thresholdvalue Dth or more as in the composite image 94 illustrated in FIG. 7,the effect processing unit 260 may make the human image 82semi-transparent by applying alpha processing to the human image 82. Inaddition, the effect processing unit 260 may add an alarm mark 88 forattention to the composite image 94 as illustrated in FIG. 7.

According to visual contact with the composite images 92 and 94, thehuman, who is the object, is intuitively determined to be in an improperdepth position and a standing position is expected to be adjusted.

When a difference between the depth of the human image and the depth ofthe foreground object is greater than or equal to the threshold valueDth, the effect processing unit 260 may apply stronger alpha processingas the difference between the depth of the human image and the depth ofthe foreground object is larger. According to this configuration, thehuman, who is the object, can recognize a degree of a difference betweena current depth position and an appropriate depth range from thestrength of the alpha processing.

In addition, although the alpha processing and the addition of the alarmmark 88 as the effect processing have been described above, the effectprocessing is not limited to these examples. For example, the effectprocessing may be a flash of the human image, contour emphasis, a huechange (for example, black-and-white forms), a blurring process,addition of a caution message, or the like.

APPLICATION EXAMPLE

Although an example in which there is one foreground object has beendescribed above, the embodiment of the present disclosure is applicableeven when there are a plurality of foreground objects. In this case, theeffect processing unit 260 may focus on only depth information of anyone foreground object or depth information of the plurality offoreground objects. When focusing on the depth information of theplurality of foreground objects, the effect processing unit 260 mayapply the effect processing if a difference between the depth of any oneforeground object and the depth of the human image is greater than orequal to the threshold value Dth or if differences between the depths ofthe plurality of foreground objects and the depth of the human image aregreater than or equal to the threshold value Dth.

(Supplement)

In addition, although an example in which the effect processing isapplied to provide notification that a human is not in an appropriatedepth range has been described above, the effect processing unit 260 mayapply the effect processing when a human image and a foreground objectin a composite image are separated by a predetermined distance.

FIG. 8 is an explanatory diagram illustrating a composite image 96 whena human image is separated from a foreground object. As in the compositeimage 96 illustrated in FIG. 8, the effect processing unit 260 may addan alarm mark 88 and an arrow image 89 for guiding movement of a humanwhen the human image 82 is separated from the foreground object 84.Thereby, the human, who is an object, is expected to move to anappropriate position.

<3. Operation of Image Processing System>

A configuration of the image processing device 20 in accordance with anembodiment of the present disclosure has been described above. Next, theoperation of the image processing system in accordance with anembodiment of the present disclosure is organized with reference to FIG.9.

FIG. 9 is a flowchart illustrating the operation of the image processingsystem in accordance with the embodiment of the present disclosure. Asillustrated in FIG. 9, first, the imaging device 10 images a human, whois an object, and acquires a captured image including RGB information(S304). In addition, the imaging device 10 also acquires depthinformation of each pixel in the captured image (S308).

The imaging device 10 recognizes an image part of the human from thecaptured image to extract a human image based on the depth informationof each pixel in the captured image (S312). Thereafter, the imagingdevice 10 transmits the extracted human image and depth information ofthe human image to the image processing device 20.

On the other hand, the image combining unit 250 of the image processingdevice 20 first draws a background image and a foreground image, forexample, read from the storage unit 230 (S316 and S320). Thereafter, theimage combining unit 250 performs processes of steps S324 to S340 forevery pixel constituting the human image in cooperation with the effectprocessing unit 260.

Specifically, the image combining unit 250 determines whether or not apixel of the foreground image corresponding to a pixel of the humanimage is a foreground object (S324). The process proceeds to the processof step S328 when the pixel of the foreground image is the foregroundobject, and the process proceeds to the process of step S332 when thepixel of the foreground image is not the foreground object.

In step S328, the image combining unit 250 determines whether or not adepth of the pixel of the human image is greater than or equal to adepth of the foreground object. When the depth of the pixel of the humanimage is greater than or equal to the depth of the foreground object,the pixel of the human image is not drawn. On the other hand, theprocess proceeds to the process of step S332 when the depth of the pixelof the human image is less than the depth of the foreground object.

In step S332, the effect processing unit 260 determines whether or not adifference between the depth of the pixel of the human image and thedepth of the foreground object is less than the threshold value Dth(S332). When the difference between the depth of the pixel of the humanimage and the depth of the foreground object is greater than or equal tothe threshold value Dth, the image combining unit 250 draws the pixel ofthe human image normally (S336). On the other hand, when the differencebetween the depth of the pixel of the human image and the depth of theforeground object is less than the threshold value Dth, the effectprocessing unit 260 applies alpha processing to the pixel of the humanimage and the image combining unit 250 draws the pixel to which thealpha processing has been applied (S340).

<4. Hardware Configuration>

Image processing by the image processing device 20 in accordance with anembodiment of the present disclosure has been described above. The imageprocessing by the image processing device 20 described above isimplemented in cooperation with hardware provided in the imageprocessing device 20 and software. Hereinafter, an example of a hardwareconfiguration of the image processing device 20 will be described withreference to FIG. 10.

FIG. 10 is an explanatory diagram illustrating the example of thehardware configuration of the image processing device 20. As illustratedin FIG. 10, the image processing device 20 includes a central processingunit (CPU) 201, a read only memory (ROM) 202, a random access memory(RAM) 203, and a host bus 204. In addition, the image processing device20 includes a bridge 205, an external bus 206, an I/F 207, an inputdevice 208, an output device 210, a storage device (hard disk drive(HDD)) 211, a drive 212, and a communication device 215.

The CPU 201 functions as an arithmetic processing unit and a controlunit and controls the general operation in the image processing device20 according to various programs. In addition, the CPU 201 may be amicroprocessor. The ROM 202 stores programs to be used by the CPU 201,arithmetic parameters, and the like. The RAM 203 temporarily storesprograms to be used in execution of the CPU 201, parameters that varyappropriately in the execution thereof, and the like. These are mutuallyconnected by the host bus 204 including a CPU bus and the like.

The host bus 204 is connected to the external bus 206 such as aperipheral component interconnect/interface (PCI) bus via the bridge205. Here, it is not necessary to separately configure the host bus 204,the bridge 205 and the external bus 206, and their functions may beimplemented in a single bus.

The input device 208 includes an input unit, which allows a user toinput information, such as a mouse, a keyboard, a touch panel, a button,a microphone, a switch, and a lever, and an input control circuit, whichgenerates an input signal based on the input by the user and outputs thesignal to the CPU 201. The user of the image processing device 20 caninput various data to the image processing device 20 or instruct theimage processing device 20 to perform a processing operation byoperating the input device 208.

The output device 210, for example, includes display devices such as acathode ray tube (CRT) display device, a liquid crystal display (LCD)device, an organic light emitting diode (OLED) device, and a lamp.Further, the output device 210 includes audio output devices such as aspeaker and a headphone. The output device 210, for example, outputsreproduced content. Specifically, the display device displays variousinformation such as reproduced video data in text or images. On theother hand, the audio output device converts reproduced audio data orthe like into audio and outputs the audio.

The storage device 211 is a device for data storage configured as anexample of a storage unit of the image processing device 20 inaccordance with this embodiment. The storage device 211 may include astorage medium, a recording device, which records data on the storagemedium, a reading device, which reads the data from the storage medium,and a deleting device, which deletes the data recorded on the storagemedium. The storage device 211, for example, includes an HDD. Thestorage device 211 drives a hard disk and stores programs to be executedby the CPU 201 and various data.

The drive 212 is a reader/writer for the storage medium and is embeddedin or externally attached to the image processing device 20. The drive212 reads information recorded on a mounted removable storage medium 24such as a magnetic disk, an optical disc, a magneto-optical disc, or asemiconductor memory and outputs the read information to the RAM 203. Inaddition, the drive 212 can write information to the removable storagemedium 24.

An example of the communication device 215 is a communication I/Fincluding a communication device and the like for establishing aconnection to a communication network 12. In addition, the communicationdevice 215 may be a wireless local area network (LAN) compatiblecommunication device, a Long Term Evolution (LTE) compatiblecommunication device or a wired communication device, which performswired communication.

<5. Conclusion>

As described above, the image processing device 20 in accordance withthe embodiment of the present disclosure executes effect processing forproviding notification that a human is not in an appropriate depth rangeif a difference between a depth of a human image and a depth of aforeground object is greater than or equal to a threshold value Dth whena foreground image including the foreground object, a background image,and the human image obtained by imaging are combined. According to thisconfiguration, notification can be provided so that it can be easilydetermined and intuitively understood whether or not the human, who isan object, is in the appropriate depth range.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

For example, the steps in processing of the imaging device 10 and theimage processing device 20 in this specification are not necessarilyprocessed in time series in the order described as the flowchart. Forexample, the steps in processing of the imaging device 10 and the imageprocessing device 20 can also be processed in an order different fromthat described as the flowchart or in parallel.

In addition, a computer program for causing hardware such as the CPU201, the ROM 202, or the RAM 203 embedded in the imaging device 10 andthe image processing device 20 to implement a function equivalent toeach configuration of the imaging device 10 and the image processingdevice 20 described above can also be created. In addition, a storagemedium storing the computer program is also provided.

Additionally, the present technology may also be configured as below.

-   (1) An image processing device including:

a depth acquisition unit configured to acquire depth information of anobject included in a captured image;

an image combining unit configured to combine the object with an imageobject having depth information according to each piece of the depthinformation; and

an effect processing unit configured to execute effect processingaccording to the depth information of the object.

-   (2) The image processing device according to (1), wherein the effect    processing unit executes the effect processing according to a    relationship between the depth information of the image object and    the depth information of the object.-   (3) The image processing device according to (2), wherein the effect    processing unit executes the effect processing when a difference    between the depth information of the object and the depth    information of the image object is greater than or equal to a    threshold value.-   (4) The image processing device according to (3), wherein the effect    processing unit executes stronger effect processing as the    difference between the depth information of the object and the depth    information of the image object is larger.-   (5) The image processing device according to any one of (1) to (4),    wherein the effect processing unit further executes the effect    processing according to a positional relationship between the image    object and the object in the composite image obtained by the image    combining unit.-   (6) The image processing device according to any one of (1) to (5),    wherein the effect processing unit executes the effect processing    for each pixel constituting the object according to depth    information of each pixel.-   (7) An image processing method including:

acquiring depth information of an object included in a captured image;

combining the object with an image object having depth informationaccording to each piece of the depth information; and

executing effect processing according to the depth information of theobject.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2011-264444 filed in theJapan Patent Office on Dec. 2, 2011, the entire content of which ishereby incorporated by reference.

What is claimed is:
 1. An image processing device comprising: a depthacquisition unit configured to acquire depth information of an objectincluded in a captured image; an image combining unit configured tocombine the object with an image object having depth informationaccording to each piece of the depth information; and an effectprocessing unit configured to execute effect processing according to thedepth information of the object.
 2. The image processing deviceaccording to claim 1, wherein the effect processing unit executes theeffect processing according to a relationship between the depthinformation of the image object and the depth information of the object.3. The image processing device according to claim 2, wherein the effectprocessing unit executes the effect processing when a difference betweenthe depth information of the object and the depth information of theimage object is greater than or equal to a threshold value.
 4. The imageprocessing device according to claim 3, wherein the effect processingunit executes stronger effect processing as the difference between thedepth information of the object and the depth information of the imageobject is larger.
 5. The image processing device according to claim 1,wherein the effect processing unit further executes the effectprocessing according to a positional relationship between the imageobject and the object in the composite image obtained by the imagecombining unit.
 6. The image processing device according to claim 1,wherein the effect processing unit executes the effect processing foreach pixel constituting the object according to depth information ofeach pixel.
 7. An image processing method comprising: acquiring depthinformation of an object included in a captured image; combining theobject with an image object having depth information according to eachpiece of the depth information; and executing effect processingaccording to the depth information of the object.